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PROBLEMS  STATEMENT 


The  proposal  identified  three  problem  areas:  nonparametric 
estimation  with  truncated  data;  asymptotic  expansions  for  randomly 
stopped  sums  (with  a  view  towards  finding  corrected  confidence 
intervals  in  sequential  problems);  and  asymptotic  local  minimaxity  in 
sequential  estimation  In  addition,  some  effort  was  devoted  to 
understanding  optimal  sequential  designs. 

SUMMARY  OF  IMPORTANT  FINDINGS 

The  problem  of  estimation  with  truncated  data  was  studied  in  the 
thesis  of  Djamel  Bellout.  He  considered  the  problem  of  searching  for 
hidden  objects  of  sizes  X^,...,X^,  say.  Let  T^  be  the  time  in  which 
the  i*'*'1  object  would  be  found  if  the  search  were  continued  forever. 
Bellout  supposed  that  the  pairs  (X^ , T^) , . . . , (X^ , TN)  are  i.i.d.,  as 
(X,T)  say,  where  X  has  a  distribution  function  F,  the  conditional 
distribution  of  T  given  X  -  x  is  exponential  with  failure  H(x) ,  and  H 
is  assumed  to  be  non-decreasing.  Here  F,  H,  and  N  may  all  be  unknown. 
If  the  process  is  observed  for  t  time  units,  where  t  >  0,  then  the 
available  data  consist  of  those  pairs  (X^,T^)  for  which  T.  <  t,  a 
biased  sample.  Bellout  was  able  to  find  nonparametric  maximum 
likelihood  estimates  of  the  parameters  F,  H,  and  N  and  show  their 
consistency.  In  addition,  he  studied  an  optimal  stopping  problem  in 
which  one  receives  a  reward,  depending  on  the  sizes  of  the  objects 
found.  He  first  derived  an  optimal  policv  for  the  case  of  known  F,  H, 
and  N,  and  then  studied  the  adaptive  policy  in  which  estimators  are 
substituted  for  the  unknown  parameters.  Two  papers  have  been  submitted 
from  this  thesis,  items  4  and  7  below. 
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Substantial  progress  was  made  on  the  problem  of  finding  asymptotic 
expansions  for  the  distributions  of  randomly  stopped  sums.  The  very 
weak  expansions,  described  in  the  proposal,  turn  out  to  be  a  general 
technique  for  determining  sequential  confidence  intervals.  Their  use 
for  setting  confidence  intervals  with  fixed  proportional  accuracy  for 
the  parameter  of  an  exponential  family  was  studied  in  Refs  2  and  3 
(below)  and  in  the  thesis  of  A.  Meslem.  The  latter  contains  a  complete 
asymptotic  solution  to  the  problem  and  extensive  numerical  calculations 
which  indicate  that  the  asymptotic  formulas  are  nearly  valid  for 
moderate  (expected)  sample  sizes.  Current  numerical  work  (still  in 
progress)  indicates  that  very  weak  expansions  may  be  quite  useful  for 
setting  confidence  intervals  after  sequential  testing. 

Asymptotic  local  minimaxity  in  sequential  estimation  was  studied 
extensively  in  the  thesis  of  Mohamed  Tahir.  Early  work  by  Woodroofe 
(1985  Ann.  Statist.,  676-688)  provided  examples  in  which  one  stopping 
time,  essentially  a  myopic  procedure,  was  asymptotically  second  order 
optimal  with  respect  to  a  large  class  of  priors.  This  may  be 
contrasted  to  the  estimation  component  of  the  problem  in  which  second 
order  optimality  requires  knowledge  of  the  prior  and,  thus,  indicates 
that  the  sensitivity  of  an  optimal  design  to  the  prior  is  less  than  the 
sensitivity  of  an  estimator.  Tahir  was  able  to  confirm  this  by 
extending  to  early  result  from  the  context  of  iso’ ated  examples  to  a 
much  more  general  one . 

During  the  period  of  the  grant,  it  was  realized  that  some  of  the 
mathematical  techniques  could  be  used  in  more  general  sequential  design 
problems,  problems  in  which  there  is  a  choice  of  experiment  at  each 
stage  as  well  as  optional  stopping.  Problems  of  this  nature  were 
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studied  in  the  theses  of  Kamal  Rekab  and  Taoufik  Zoubeidi.  While 
limited  to  special  cases,  the  results  of  these  theses  are  encouraging. 
They  indicate  that  a  natural  analogue  of  the  myopic  procedure  (for 
stopping  problems)  may  enjoy  the  same  asymptotic  optimality  properties 
in  more  general  design  problems. 


PERSONNEL  SUPPORTED  BY  THE  GRANT 

Michael  Woodroofe;  project  director  and  principle  investigator. 
Martha  Aliaga;  graduate  student. 

Djamel  Bellout;  graduate  student. 

Ki-Heon  Choi;  graduate  student. 

A.  Meslem;  graduate  student. 

Kamal  Rekab;  graduate  student. 

Taoufik  Zoubeidi;  graduate  student. 
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which  were  prepared  with  the  support  of  the  Grant.  Abstracts  of  these 
papers  may  be  found  in  Appendix  1 . 

1.  Woodroofe,  M.  (1987).  Confidence  intervals  with  fixed  proportional 
accuracy.  J.  Statist.  Plan.  Inf.,  vol.  15,  131-1A6. 

2.  Meslem,  A.  (1988).  Asymptotic  expansions  for  fixed  width 
confidence  intervals.  J.  Statist.  Plan.  Inf.,  vol.  17,  51-66.  By 
A.  Meslem. 
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3.  Woodroofe,  M.  (1988).  Fixed  proportional  accuracy  in  three  stages. 
Proceedings  of  the  Fourth  Purdue  Symposium  on  Statistical  Decision 
Theory  and  Related  Topics,  vol.  2,  209-222. 

d.  Bellout,  D.  (1988).  Order  restricted  estimation  of  distributions 
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Analysis . 
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Choi,  Ki-Heon.  An  Adaptive  Sequential  Probability  Ratio  Test  for 
Autoregressive  Process. 
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intervals  with  fixed  proportional  accuracy . 
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CONFIDENCE  INTERVALS  WITH  FIXED  PROPORTIONAL 
ACCURACY* 

Michael  WOODROOFE 

Department  of  Statistics.  University  of  Michigan.  Ann  Arbor,  \11  48109,  USA 

Revised  1'  July  1 SJ85;  reused  version  received  2  December  1985 
Recommended  by  M.L.  Puri 

Abstract :  The  problem  of  setting  a  confidence  interval  with  fixed  proportional  accuracy  for  the 
mean  of  a  normal  distribution  is  considered.  A  sequential  procedure  is  proposed;  and  asymptotic 
expansions  for  its  average  coverage  probabilities  and  its  expected  sample  size  are  obtained.  The 
procedure  is  shown  to  have  asymptotically  minimal  expected  sample  size,  subject  to  a  constraint 
on  the  error  probabilities. 

A  MS  Subject  Classification:  62L12. 

Key  woras  and  phrases:  Stopping  times;  Average  confidence  levels;  Posterior  distributions; 
Asymptotic  expansions;  Sequential  decision  problem. 


1.  Introduction 

The  possibility  of  using  sequential  methods  to  set  a  Fixed  width  confidence  inter¬ 
val  for  the  mean  of  a  distribution  with  an  unknown  variance  has  attracted  substan¬ 
tial  interest  in  theoretical  statistics.  This  interest  was  stimulated  by  Stein’s  (1945) 
two-stage  procedure,  the  fully  sequential  procedures  of  Anscombe  (1953)  and  Chow 
and  Robbm'  (1965)  for  the  normal  and  non-parametric  cases,  and  Hall’s  (1981) 
three  stage  procedure.  The  recent  paper  by  Finster  (1985)  describes  multiparameter 
extensions  and  may  be  consulted  for  further  references. 

Much  of  this  research  exploits  the  independence  of  the  sample  mean  and  variance 
in  the  normal  case.  The  independence  provides  a  simple  relation  between  the 
coverage  probability  of  a  sequential  procedure  and  the  first  two  moments  of  the 
stopping  time;  and  this  relation  permits  a  second  order  asymptotic  expansion  for  the 
coverage  probability  as  the  width  of  the  interval  shrinks  to  zero.  See,  for  example, 
Nnscombe  (1953),  Hall  (1981),  and  Woodroofe  (1982,  Section  10.2).  The  second 
order  terms  in  these  expansions  are  especially  interesting,  since  the  effect  of  optional 
stopping  appears  there. 


•  Research  supported  by  the  U.S.  Army  under  DAAG29-85-K-0008. 
0378-3758/87'$3.50  ;  1987,  Elsevier  Science  Publishers  B.V.  (North-Holland) 
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ASYMPTOTIC  EXPANSIONS  FOR  FIXED  WIDTH  CONFIDENCE  INTERVAL 


By 

Abd-El-Hakim  Meslem' 
April  1986 

The  University  of  Michigan 


Abstract 

A  sequential  procedure  for  setting  a  fixed  width  confidence  interval  for  the  natural  pa¬ 
rameter  of  an  exponential  family  is  proposed  and  studied.  Average  confidence  levels  are 
considered  instead  of  the  classical  confidence  curves.  The  procedure  is  shown  to  perform 
very  well  in  a  simulation  study. 


Key  words  and  phrases:  stopping  time:  average  coverage  probability:  Bavsian  model; 
excess  over  the  boundary:  asymptotic  expansions. 

AMS  (1980)  classification:  62L12 

Research  supported  by  the  US.  Army  under  DA  AG  29-85-K-0008 
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FIXED  PROPORTIONAL  ACCURACY  IN  THREE  STAGES 


Michael  Woodroofe 

Department  of  Statistics 
The  University  of  Michigan 
Ann  Arbor,  Michigan  48109,  U.S.A. 


1.  INTRODUCTION 

Let  xi,X2,...  denote  independent  random  variables  which  are  normally 
distributed  with  an  unknown  mean  8  €  R  =  (  —  00,00)  and  unit  variance;  let  Q 
denote  a  positive,  twice  continuously  differentiable  function  on  R.  A  sequential 
confidence  interval  for  8  with  width  proportional  to  \/Q(8)  is  sought;  that  is, 
given  0  <  7  <  1  and  h  >  0,  a  stopping  time  t  =  t(i1,x 2,...)  and  an  estimator 
9t  =  9t(x  1, . . . ,  xt)  are  sought  for  which 

P,{\  8t  -  8  {<  hy/Q(8)}  *  ^  (1) 

for  all  9.  Small  values  of  h  are  of  special  interest  here,  and  k  is  interpreted 
to  mean  equality  up  to  Woodroofe  (1926a)  considered  this  problem 

and  established  (1)  in  a  very  weak  sense  for  a  fully  sequential  procedure,  one 
which  monitors  the  data  continuously.  The  main  result  of  this  paper  is  to  show 
that  (1)  holds  (in  a  conventional  sense)  for  a  three  stage  procedure,  one  which 
requires  monitoring  the  data  only  three  times.  For  simplicity,  8  is  estimated  by 
the  sample  mean, 

8n  =*„  =  (*,  +...  +  X„)/n,  n  =  1,2, ... ,  (2) 

although  more  general  estimators  are  amenable  to  similar  analyses,  as  in 
Woodroofe  (1986a).  Thus,  the  problem  is  to  find  a  stopping  time  t  for  which 
(I)  holds  with  8t  as  in  (2). 

To  get  a  rough  idea  of  the  sample  size  required,  let  $  denote  the  standard 
normal  distribution  function.  If  a  sample  of  fixed  (non-random)  size  n  were 
taken,  then  the  left  side  of  (1)  would  be  2$(/i-l/[nQ(ff)]}  —  1,  which  exceeds 

n  >  N  :=  c1  /h}Q(6),  (3) 

where  e  =  $~l[(I  +  *y) /2),  and  :=  indicates  a  definition.  Of  course,  N  depends 
on  8,  which  is  unknown. 

To  describe  the  sequential  procedure,  let  q  denote  a  twice  continuously 
differentiable  function  on  let 

Qn  =  max{l/n,  min{rt, Q(^fn)}} 
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ORDER  RESTRICTED  ESTIMATION  OF 
DISTRIBUTIONS  WITH  CENSORED  D/TA 


by 


DJANIEL  BELLOUT* 
September  1986 
The  University  of  Michigan 


Abstract 

Consider  a  finite  population  of  hidden  objects,  and  search  for  these  objects  for  one  unit 
of  time.  Supoose  that  both  the  size  and  the  discovery  time  of  the  objects  have  unknown 
distributions,  and  that  the  conditional  distribution  of  time  given  size  is  exponential  with, 
as  parameter,  an  unknown  non-negative  and  non-decreasing  function  of  the  size.  Order 
restricted  M.L.E.'s  are  derived  for  this  function,  other  parameters  are  estimated,  and  the 
consistency  of  the  estimates  is  shown. 


Key  u  rds  a.i i  phrases :  nonparametnc,  maximum  likelihood  estimation,  order  restric¬ 
tion,  isotonic  regression,  least  concave  majorant,  consistency. 

A  MS  198C  subject  classifications:  primary  62G05;  secondary  62G30. 

‘Research  supported  by  the  U.S.  Army  under  DAAG  29-85-K-0008. 
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ASYMPTOTICALLY  OPTIMAL  BAYESIAN  SEQUENTIAL 
POINT  ESTIMATION  WITH  CENSORED  DATA 


Mohamed  Tahir 

Department  of  Statistics 
The  University  of  Michigan 
Ann  Arbor,  MI  48109 


Key  Words  and  Phrases:  posterior  expected  loss;  Bayes  estimator;  Markov  processes; 
infinitesimal  operator ;  Dynkm’s  identity;  asymptotic  expansions. 


ABSTRACT 

A  stopping  time  for  estimating  the  failure  rate  of  an  exponential  distribution  with  a 
smooth  loss  function  and  a  gamma  prior  is  constructed  using  censored  data.  The  optimality 
of  this  stopping  time  is  established  by  using  Dynkin’s  identity  for  Markov  processes. 
Although  the  results  obtained  are  asymptotic  in  one  sense,  the  optimality  is  exact. 


1.  INTRODUCTION 

Let  Afj,  ...,  Xu  be  conditionally  independent  and  exponentially  distributed  random  vari¬ 
ables  with  common  failure  rate  u,  given  that  W  =  where  W  is  a  random  variable  having  a 
gamma  distribution  with  parameters  a0  and  b0.  Further,  let 

M{t)=  £  V<1  (*) 

«=1 

denote  the  number  of  observations  which  have  been  made  by  time  t  >  0,  where  I  denotes  the 
indicator  function.  Also,  let  F,  be  the  sigma-algebra  generated  by  M($),  0  <  s  <  t  and 

Y . .  where  Yh  ...,  Y ^  are  the  order  statistics  of  Xv  ...,XN.  It  follows  that  the 

likelihood  function  at  time  t  is  proportional  to 

MU) 

l{oj  1 1 )  oc  exp{-  oj  Yi  -  oj  [N  -  M{t)\t) 

»=i 

for  oj  >  0.  Thus,  the  posterior  distribution  of  W,  given  F, ,  is  a  gamma  distribution  with 
new  parameters 
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CORRECTED  CONFIDENCE  LEVELS 


FOR 

ADAPTIVELY  DESIGNED  EXPERIMENTS 


By 

* 

Michael  Woodroofe 
The  University  of  Michigan 

ABSTRACT 

« 

Consider  a  non-linear  regression  model  y  =  g(x  ;u>)  -  e  . 

K  K  K 

k  =  1,2 .  in  which  the  design  variables,  * 

x^Yj . may  be  functions  the  previous  responses.  A 

very  weak  asymptotic  expansion  for  the  distribution  of  the 
maximum  likelihood  estimator  is  presented.  from  which  the 
primary  effect  of  the  adaptive  nature  of  the  design  may  be 
seen . 


Key  words  and  phrases:  non-linear  regression:  stable  designs: 
martingale  central  limit  theorem;  very  weak  expansions. 

*  Research  supported  by  the  U.S.  Army  Research  Office  under 
DAAG  29-85-0008. 
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AN  ADAPTIVE  RULE  BASED  ON  DISTRIBUTION 
ESTIMATES  FROM  TRUNCATED  DATA 


Djamel  Bellout* 

Department  of  Statistics/C  L  IS 
The  George  Washington  University 
Washington,  D.  C.  20052 


Key  words  and  phrases:  payoff;  Markov  process;  characteristic  operator;  stopping 
time;  adaptive  rule. 


ABSTRACT 


Consider  a  finite  population,  of  large  but  unknown  size,  of  hidden  objects.  Con¬ 
sider  searching  for  these  objects  for  a  period  of  time,  at  a  certain  cost,  and  receiving 
a  reward  depending  on  the  sizes  of  the  objects  found.  Suppose  that  the  size  and 
discovery  time  of  the  objects  both  have  unknown  distributions,  but  the  conditional 
distribution  of  time  given  size  is  exponential  with  an  unknown  non-negative  and 
non-decreasing  function  of  the  size  as  failure  rate.  The  goal  is  to  find  an  optimal 
way  to  stop  the  discovery  process.  First,  assuming  that  the  above  parameters  are 
known,  an  optimal  stopping  time  is  derived  and  its  asymptotic  properties  are  stud¬ 
ied.  Then,  an  adaptive  rule,  based  on  order  restricted  estimates  of  the  distributions 
from  truncated  data,  is  presented.  This  adaptive  rule  is  shown  to  perform  nearly  as 
well  as  the  optimal  stopping  time,  for  large  population  size. 


L  INTRODUCTION 


Consider  a  finite  population,  of  large  but  unknown  size  iV,  of  hidden  objects. 
Consider  searching  for  these  objects  for  a  period  t  >  0  of  time,  at  a  certain  cost  pro- 
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ABSTRACT 

In  a  clinical  trial  of  two  treatments,  one  goal  of  the  experimenter  is  to  design 
an  experiment  such  that  the  number  of  patients  assigned  to  the  inferior  treatment  is 
minimized.  A  Bayesian  formulation  for  this  problem  is  studied.  Treatment  outcomes 
are  assumed  to  be  independent  and  normally  distributed,  with  conjugate  priors.  The 
analysis  is  conducted  in  a  large  sample  limit  where  sampling  costs  are  scaled  to 
zero.  First  we  show  there  is  little  harm  restricting  attention  to  procedures  that  stop 
according  to  Kiefer  and  Sacks  stopping  rule,  there  are  procedures  in  this  class  with 
risks  that  exceed  the  Bayes  risk  by  the  cost  of  a  fixed  number  of  observations.  Using 
this  we  then  develop  fully  efficient  procedures  with  risks  asymptotic  to  the  Bayes  risk 
in  our  large  sample  limit. 
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ORDER  RESTRICTED  ESTIMATION  OF  DISTRIBUTIONS  WITH  CENSORED 
DATA  AND  APPLICATION  TO  A  STOPPING  PROBLEM 


by 

Djamel  Bellout 


Chairperson:  Michael  B.  Woodroofe 


Consider  a  finite  population  of  hidden  objects  with  sizes  X\,  X2, . . .  ,  Xyv-  Let  T,  denote 
the  time  at  which  the  t-th  object  would  be  found  in  an  infinite  search.  Let  F  be  the 
unknown  distribution  of  X;  GT  the  conditional  distribution  of  T  given  X.  Suppose  that 
G"  is  of  the  form  G'(f|z:)  =  1  -  exp{ -*#(*)},  where  H  is  an  unknown,  non-negative,  and 
non-decreasing  function.  Finally,  suppose  that  only  the  pairs  (AT,,  T,j  for  which  T,  <  t,  t  >  0 
are  observed.  An  optimal  stopping  time  is  constructed  for  the  stopping  problem  with  the 
payoff  Ztt(t)  =  w(A|)/{  T,  <  t}  -  ct,  where  u  is  a  non-decreasing  function  and  c  is  the 

cost  of  sampling  per  unit  time,  when  F,  H  and  N  are  known.  Maximum  likelihood  estimates 
of  F,  H  and  N  are  obtained,  using  techniques  of  isotonic  regression.  These  are  shown  to  be 
strongly  consistent,  and  some  remarks  are  made  about  their  asymptotic  distribution.  An 
adaptive  stopping  time  in  which  F,  H  and  N  are  replaced  by  estimates  is  proposed.  The 
adaptive  rule  is  shown  to  perform  as  well  as  the  optimal  stopping  rule,  asymptotically. 
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AN  ADAPTIVE  SEQUENTIAL  PROBABILITY  RATIO  TEST  FOR 
AUTOREGRESSIVE  PROCESS 


by 

Ki-Heon 


Choi 


Chairperson:  Michael  B.  Woodroofe 


Consider  the  first  order  autoregressive  model 

Xn  =  6  +  pXn- 1  +  en>  n  £  Z  • 

where  en,  n  £  Z,  axe  i.i.d.  normally  distributed  random  variables  with  mean  0 
and  variance  a2.  Here  cr2,  6,  and  p  are  parameters,  we  consider  the  problem  of 
sequentially  testing  hypotheses  about  a  parameter  6  in  the  presence  of  the  nuisance 
parameter  p  . 

The  sequential  testing  problem  leads  directly  to  the  study  of  stopping  times  of 
the  form 

t  =  ttt  =  inf{n  >  1  :  5n  >  a  } 

where  Sn  is  a  function  of  X\,  •••, Xn  and  a  >  0  ,  and  the  problem  of  evaluating 
of  conditional  boundary  crossing  probabilities  of  the  form 

•v, 

Pr(ta  >n  |S„  =  o  +  r) 


for  a  ,  t  >  0  and  n  =  1,  2,  •  •  •.  The  approach  is  to  compute  the  conditional  probabil¬ 
ity  of  crossing  the  boundary  a  ,  looking  backward  along  the  sequence  Sn_i,  •  •  ■  -Sn-k  > 
by  obtaining  local  limit  theorems  for  the  distribution  of  5n,  Sn- 1,  •  •  ■ ,  Sn_ *  . 
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ASYMPTOTICALLY  OPTIMAL  BAYESIAN  AND  MINIMAX  SEQUENTIAL  POINT 

ESTIMATION 


by 

Mohamed  Tahir 


Chairperson:  Michael  B.  Woodroofe 

Four  related  problems  are  considered  in  this  thesis.  The  first  problem  consists  of  deter¬ 
mining  an  optimal  stopping  time  for  estimating  the  failure  rate  of  an  exponential  distribution 
with  a  general,  smooth  loss  function  and  a  gamma  prior,  using  censored  data.  The  con¬ 
struction  of  such  a  stopping  time  involves  obtaining  asymptotic  expansions  for  the  posterior 
expected  loss,  computing  the  infinitesimal  operator  of  some  resulting  Markov  process,  and 
using  Dynkin’s  identity.  The  second  problem  is  to  obtain  an  asymptotic  lower  bound  for 
the  minimax  regret  of  a  sequential  estimation  procedure,  in  the  case  of  a  one-parameter 
exponential  family,  for  a  general  class  of  estimators  of  the  population  mean  and  with  a 
general,  smooth  loss  function.  The  program  is  to  determine  the  limit  of  the  Bayes  regret 
and  then  maximize  with  respect  to  the  prior  distribution.  The  third  problem  is  to  develop 
a  uniform  version  of  the  non-linear  renewed  theorem.  Finally,  the  fourth  problem  is  to  find 
a  stopping  time  which  attains  the  asymptotic  lower  bound  of  the  second  problem.  The 
procedure  used  to  yield  such  a  stopping  time  requires  the  results  developed  in  the  third 
problem. 
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ABSTRACT 

ASYMPTOTIC  EFFICIENCY  IN  SEQUENTIAL  DESIGNS  FOR 

ESTIMATION 

by 

Kamel  Rekab 

Chairperson:  Michael  B.  Woodroofe 

Properties  of  a  sequential  design  for  nonlinear  regression  and  related  problems  are 
investigated.  The  mam  results  determine  an  asymptotic  lower  bound  for  the  Bayes 
risk,  and  a  fully  sequential  procedure  as  well  as  a  two  stage  procedure  that  are  first 
order  efficient. 


21 


ABSTRACT 


SEQUENTIAL  TESTS  FOR  CLINICAL  TRIALS  OF  TWO 
TREATMENTS  WITH  BOUNDED  DEFICIENCY 


by 

Taoufik  Zoubeidi 


co- Chairpersons:  Robert  W  Keener  and  Michael  B.  Woodroofe 

In  a  clinical  trial  of  two  treatments,  one  goal  of  the  experimenter  is  to  design 
an  experiment  such  that  the  number  of  patients  assigned  to  the  inferior  treatment  is 
minimized.  A  Bayesian  formulation  for  this  problem  is  studied.  Treatment  outcomes 
are  assumed  to  be  independent  and  normally  distributed,  with  conjugate  pnors.  The 
analysis  is  conducted  in  a  large  sample  limit  where  sampling  costs  are  scaled  to 
zero.  First  we  show  there  is  little  harm  restricting  attention  to  procedures  that 
stop  according  to  Kiefer  and  Sacks  stopping  rule.  There  are  procedures  in  this  class 
with  risks  that  exceed  the  minimum  Bayes  risk  by  the  cost  of  a  fixed  number  of 
observations.  Using  this  we  then  develop  procedures  with  bounded  deficiency.  A 
Monte  Carlo  study  shows  that  these  procedures  perform  better  than  the  pairwise 
procedure. 
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